Wherefore, what is claimed is: 



1 . An automated system for capturing and viewing an event having 
event participants, comprising: 

multiple cameras simultaneously capturing images of different sub-events 
occurring in a space associated with an event; 

a server capable of recording and broadcasting the captured sub-events; 

and 

one or more clients in network connection with said server that view 
portions of the captured event. 

2. The system of Claim 1 , wherein the multiple cameras are one of: 

a 360-degree camera centrally positioned to monitor in substantially 360- 
degrees the space in which the event occurs; 

a remote view camera positioned so as to capture a view of event 
participants in said space associated with said event to be transmitted to a client 
over said network; 

a presenter view camera positioned so as to capture a view of an 
overview of the space associated with the event; and 

a whiteboard capture camera positioned so as to capture strokes written 
on a whiteboard. 

3. The system of Claim 2 wherein said omni-directional camera 
comprises a set of cameras configured in a circular back to back fashion. 
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4. The system of Claim 3 further comprising a panoramic stitcher that 
stitches together images captured from each camera to create a panoramic 
image of said space in which the event occurs. 

5 

5. The system of Claim 2 wherein said presenter view camera is 
integrated with a microphone. 

6. The system of Claim 2 wherein said remote view camera is 
10 integrated with a microphone. 

7. The system of Claim 2 further comprising a virtual director that 
allows switching between said multiple cameras to display a view of one of said 
different sub-events. 

15 

8. The system of Claim 7 wherein said virtual director determines 
which camera view to display by: 

determining if a person is speaking and facing toward a display that 
displays at least one remote event participant, and if so using a camera view 
20 captured by said remote camera to display; 

determining if a person is talking and the presenter view camera 
can track them and provide a higher resolution image than the 360-degree 
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camera, and if so using a camera view captured by said presenter view camera 
for display; and 

else, using a camera view captured by said 360-degree camera to 

display. 

9. The system of Claim 1 further comprising a microphone device that . 
simultaneously captures audio data of said event. 

10. The system of Claim 9 wherein said microphone device is used to 
determine the direction from which an event participant is speaking by using 
sound source localization. 

1 1 . The system of Claim 9 wherein captured images of the sub-events 
are used in combination with sound source localization to refine the determined 
direction from which an event participant is speaking. 

12. The system of Claim 1 wherein said system is used for at least one 

of: 

broadcasting an event to one or more remote clients; 
receiving data associated with the event from a client; 
recording said event; and 
browsing of a recording of said event. 
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13. The system of Claim 1 further comprising a projector for projecting 
event materials onto a screen. 

1 4. The system of Claim 1 further comprising a monitor for displaying 
one or more remote participants or other meeting materials in said space where 
said event occurs. 

15. The system of Claim 1 further comprising an event kiosk which is 
used to control one of event broadcast, recording and recorded event playback. 

16. The system of Claim 15 wherein said kiosk further comprises a 
graphical user interface. 

17. The system of Claim 16 wherein said graphical user interface 
comprises at least one of: 

an initial display showing initial status of the system; 

a setup display for allowing a user to start recording of an event; 

an event status display displaying images from said multiple cameras, 
event duration and a control for stopping or canceling said recording of said 
event; and 

a stop meeting confirmation display that confirms the user's action of 
stopping or canceling said recording of said event. 
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18. The system of Claim 15 further wherein said event kiosk is located 
on one of said multiple cameras. 

19. The system of Claim 1 further comprising a graphics capture 
device used to capture data presented in said event or transferred between the 
multiple cameras, server and one more clients. 

20. The system of Claim 1 further comprising an archive server on 
which recorded events are stored and wherein said archive server plays back 
said recorded events to said clients. 

21 . The system of Claim 1 further comprising an archive server on 
which annotations to said captured sub-events are saved. 

22. A computer-implemented process for recording an event, 
comprising the process actions of: 

recording images captured by multiple different cameras of different types 
simultaneously capturing different portions of an event; 

recording audio synchronized with said recorded images; 

storing said images and synchronized audio as event data on a server 
capable of distributing the said event data. 
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23. The computer-implemented process of Claim 22 further comprising 
the process action of said server transmitting said event data to one or more 
clients in network connection with said server. 

24. The computer-implemented process of Claim 23 further comprising 
the process action of said one or more clients playing back said stored images 
and synchronized audio. 

25. The computer-implemented process of Claim 23 wherein said 
multiple cameras are at least one of: 

a 360-degree camera centrally positioned to monitor in substantially 360- 
degrees the space in which the event occurs; 

a remote view camera positioned so as to capture a view of event 
participants in the space in which the event occurs to be transmitted to a viewer 
over said network; 

a presenter view camera positioned so as to capture an overview of the 
space in which said event occurs; and 

a whiteboard capture camera positioned so as to capture strokes written 
on a whiteboard. 

26. The computer-implemented process of Claim 22 further comprising 
the process action of capturing graphics data of said event. 
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27. The computer-implemented process of Claim 26 further comprising 
the process action of determining key frames that summarize event sub-events 
by analyzing said captured graphics data. 

28. The computer-implemented process of Claim 27 further comprising 
a process action of indexing said stored event data, wherein said indexing 
process action comprises determining graphics key frames that summarize sub- 
events of said event by analyzing said images captured by a graphics capture 
device and using said key frames as an index to the corresponding portion of 
audio and images of the stored event data. 

29. The computer-implemented process of Claim 25 further comprising 
the process action of indexing said stored event data by at least one of the 
following process actions: 

determining at least one whiteboard key frame that summarizes a sub- 
event by analyzing said images captured by said whiteboard camera and using 
said whiteboard key frames as an index to the corresponding portion of audio 
and images of the meeting; 

determining a timeline outlining the sequence of sub-events occurring in 
the event and using said timeline as an index to a specific portion of the audio or 
images of the event; and 

using speech recognition to transcribe the audio of the event and using 
word searches to locate specific portions of said audio. 
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30. The computer-implemented process of Claim 29 wherein said word 
searches are by phonetic spelling. 

31 . The computer-implemented process of Claim 22 further comprising 
the process action of indexing said stored event data for playback. 

32. The computer-implemented process of Claim 31 wherein the 
process action of indexing said stored event data comprises determining who is 
speaking by analyzing audio of said, event and segmenting the audio of each 
speaker to be accessible on an individual basis to be used as an index of said 
stored event data. 

33. The computer-implemented process of Claim 31 wherein the 

15 process action of indexing said recorded event data comprises analyzing said 
event audio to categorize tone of voice for event participants speaking at said 
event and indexing said stored event data by tone of voice. 

34. The computer-implemented process of Claim 31 wherein the 

20 process action of indexing said stored event data comprises determining speaker 
turn rate, wherein speaker turn rate comprises the number of speakers of a given 
period of time, and using said speaker turn rate to index said stored event data. 
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35. The computer-implemented process of Claim 31 wherein the 
process action of indexing said stored event data comprises determining when 
event recording was turned on and off using event turn on and turn off events to 
index said stored event data. 

36. The computer-implemented process of Claim 31 wherein the 
process action of indexing said stored event data comprises determining when a 
shared application is started or stopped and using this to index said stored event 
data. 

37. The computer-implemented process of Claim 31 wherein the 
process action of indexing said stored event data comprises determining when 
someone is pointing at an object and using the pointing event to index to said 
stored event data. 

38. The computer-implemented process of Claim 31 wherein the 
process action of indexing said stored event data comprises using data recorded 
by a client to be used as an index to said recorded meeting data. 

39. The computer-implemented process of Claim 24 further comprising 
the process action of adjusting meeting playback speed using a time 
compression feature, wherein said time compression feature appropriately slows 
down or speeds up playback of the audio signal. 
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40. The computer-implemented process of Claim 39 wherein said time 
compression feature appropriately slows down or speeds up playback of an 
image sequence that is synchronized with said audio signal. 

41 . The computer-implemented process of Claim 39 wherein said time 
compression feature is applied to the audio signal of a single speaker. 

42. The computer-implemented process of Claim 24 further comprising 
the process action of segmenting the audio from a given speaker in said event. 

43. The computer-implemented process of Claim 42 further comprising 
the process action of providing a fast forward function that allows skipping of a 
portion of the audio of one or more speakers. 

44. A graphical user interface for viewing and playing back event data 
comprising: 

a speaker image display pane displaying a speaker speaking at an event; 
a whiteboard display pane displaying an image of content written on a 
whiteboard; 

and a panoramic image display pane displaying a panoramic image of 
said event. 
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45. The graphical user interface of Claim 44 further comprising a key 
frame image summarizing content written on a whiteboard. 

46. The graphical user interface of Claim 45 further comprising a 
thumbnail view of said key frame, wherein selecting said thumbnail view will 
display said key frame in said whiteboard pane. 

47. The graphical user interface of Claim 44 further comprising a 
graphics display pane displaying an image captured by a graphical capture 
device. 

48. The graphical user interface of Claim 47 further comprising a key 
frame image summarizing content of said data captured by said graphical 
capture device. 

49. The graphical user interface of Claim 47 wherein said key frame is 
an electronic slide of an electronic presentation. 

50. The graphical user interface of Claim 48 wherein a user can jump 
to a starting point of at least one of audio or video corresponding to the key 
frame image in a meeting playback timeline by selecting said key frame 
thumbnail. 



61 



51 . A system for conducting a distributed meeting, the system 
comprising: 

a 360-degree camera for capturing images of meeting participants in a 
meeting in substantially 360 degrees about said 360-degree camera; 

a whiteboard camera for capturing images of contents written on a 
whiteboard; 

a presenter camera for capturing images of an overview of the meeting 

room; 

a microphone array for capturing the audio of the meeting that is 
synchronized with one of said images captured by said 360-degree camera, 
whiteboard camera or presenter camera; and 

a meeting server for performing processing required to broadcast and 
record meeting data. 

52. The system of Claim 51 further comprising a network connecting 
said meeting server to at least one remote meeting participant, wherein said 
network is used to broadcast meeting images and audio from said server to 
participants and receive audio and images from said remote meeting participants 
at said server. 

53. The system of Claim 51 further comprising an archive server for 
performing processing required to playback recorded meeting data. 
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54. The system of Claim 53 further comprising one or more archive 
clients capable of playing back said captured images and synchronized audio. 

55. An automated system for capturing and viewing an event having 
event participants, comprising: 

multiple cameras simultaneously capturing images of different sub-events 
occurring in a space associated with an event; 

an event server, that processes in substantially real time said event data; 

an event post processor that process said event data only when the event 
is completed; and 

at least one event client in connection with said event server wherein said 
event client allows viewing live events and archived events. 

56. The automated system of Claim 55 further comprising an archive 
server which acts as a store for said event data. 

57. The automated system of Claim 55 wherein the event server 
performs at least one of the following functions: 

acquiring audio or video from said capture devices; 

compressing said audio or video; 

archiving said audio or video; 

providing video or audio to said client; and 
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receiving audio or video generated at said event client from said event 

client. 



58. A computer-readable medium having computer-executable 

5 instructions for viewing a recorded event, said computer-executable instructions 
comprising: 

simultaneously capturing images of different sub-events by of an event 
with multiple cameras each capturing a different sub-event; 

capturing audio associated with the different sub-events; and 
10 transmitting the captured sub-events and associated audio from a server 

to one or more clients in network connection with said server. 

59. The computer-readable medium of Claim 58 further comprising a 
computer-executable instruction for establishing a two-way connection between 

15 said server and said one or more clients for transferring images, audio and data 
between said server and said one or more clients. 

60. The computer-readable medium of Claim 58 further comprising a 
computer-executable instruction for projecting event data onto a screen. 

20 

61 . The computer-readable medium of Claim 60 wherein said event 
data that is projected onto a screen includes a view of one or more clients' 
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meeting participants, a view of the event captured by one of said multiple 
cameras and a software application display . 

62. A computer-implemented process for viewing an event comprising 
the process actions of: 

simultaneously capturing images of different sub-events of an event with 
multiple cameras each capturing a different sub-event; 

capturing audio associated with the different sub-events synchronized with 
said captured images; 

storing said captured images and associated audio on a server; and 

viewing the captured sub-events and associated audio by one or more 
clients in network connection with said server. 

63. The computer-implemented process of Claim 62 further comprising 
a process action of indexing said event data, wherein said indexing process 
action comprises determining graphics key frames that summarize sub-events of 
said event by analyzing images captured by a graphics capture device and using 
said key frames as an index to jump to the corresponding portion of audio and 
images of the event data while the event is ongoing. 

64. The computer-implemented process of Claim 62 further comprising 
a process action of indexing said event data, wherein said indexing process 
action comprises determining at least one whiteboard data key frame that 
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summarizes a sub-event by analyzing said images captured by one of said 
multiple cameras and using said whiteboard key frames as an index to jump to 
the corresponding portion of audio and images of the meeting while the event is 
ongoing. 

65. The computer-implemented process of Claim 62 further comprising 
the process action of playing back a portion of said event while said event is 
ongoing using a time compression feature, wherein said time compression 
feature appropriately slows down or speeds up playback of the audio signal. 

66. The computer-implemented process of Claim 65 wherein said time 
compression feature appropriately slows down or speeds up playback of an 
image sequence that is synchronized with said audio signal. 

67. The computer-implemented process of Claim 65 wherein said time 
compression feature is applied to the audio signal of a single speaker. 

68. The computer-implemented process of Claim 65 further comprising 
the process action of providing a fast forward function that allows skipping of a 
portion of the audio of one or more speakers. 

69. A system for conducting a distributed meeting, the system 
comprising: 
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a 360-degree camera for capturing images of meeting participants in a 
meeting room in substantially 360 degrees about said 360-degree camera, 
wherein said 360-degree camera includes an integrated computer that performs 
processing required to broadcast said images and associated meeting data. 

70. The system of Claim 69 further comprising at least one of: 
a whiteboard camera for capturing images of contents written on a 

whiteboard; and 

a presenter view camera for capturing images of an overview of the 
meeting room. 

71 . The system of Claim 70 further comprising a microphone array for 
capturing the audio of the meeting that is synchronized with one of said images 
captured by said 360-degree camera, whiteboard camera or presenter view 
camera. 

72. The system of Claim 69 wherein said associated meeting data 
comprises audio that is synchronized with said images. 
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